The use of synchrony in parent-child interaction can be measured on a signal-level

In our approach, we aim at an objective measurement of synchrony in multimodal tutoring behavior. The use of signal correlation provides a well formalized method that yields gradual information about the degree of synchrony. For our analysis, we used and extended an algorithm proposed by Hershey & Movellan (2000) that correlates single-pixel values of a video signal with the loudness of the corresponding audio track over time. The results of all pixels are integrated over the video to achieve a scalar estimate of synchrony

Exploring "Associative Talk": When German Mothers Instruct Their Two Year Olds about Spatial Tasks

In this study, maternal input was analyzed during a task, in which German mothers instructed their two-year-old children to put two objects together in a particular way. In the setting, the spatial relation (ON and UNDER) and the canonicality of these relations (canonical such as ‘a pot on a table’ and noncanonical like ‘a train on a tunnel’) were varied. Two kinds of discourse strategies are proposed that characterize mothers’ input in this task: bring-in and follow-in. For the analysis, an automatic procedure was developed, in which the amount of words spent on a strategy was related to the overall word amount. The data suggest that the canonicality of the task can change the discourse: Bring-in strategies dominated the discourse in tasks with canonical spatial relations while in more difficult tasks with non-canonical relations, German-speaking mothers used follow-ins significantly more often than in the canonical tasks. Together, the results of this study shed light on the process of an on-line adaptation of the mother to her child and give us insight into how a situated understanding in a task-oriented discourse emerges

A competitive mechanism for self-organized learning of sensorimotor mappings

Hemion N, Joublin F, Rohlfing K. A competitive mechanism for self-organized learning of sensorimotor mappings. In: 2011 IEEE International Conference on Development and Learning (ICDL). IEEE; 2011

Verbs in mothers’ input to six-month-olds:synchrony between presentation, meaning, and actions is related to later verb acquisition

In embodied theories on language, it is widely accepted that experience in acting generates an expectation of this action when hearing the word for it. However, how this expectation emerges during language acquisition is still not well understood. Assuming that the intermodal presentation of information facilitates perception, prior research had suggested that early in infancy, mothers perform their actions in temporal synchrony with language. Further research revealed that this synchrony is a form of multimodal responsive behavior related to the child’s later language development. Expanding on these findings, this article explores the relationship between action–language synchrony and the acquisition of verbs. Using qualitative and quantitative methods, we analyzed the coordination of verbs and action in mothers’ input to six-month-old infants and related these maternal strategies to the infants’ later production of verbs. We found that the verbs used by mothers in these early interactions were tightly coordinated with the ongoing action and very frequently responsive to infant actions. It is concluded that use of these multimodal strategies could significantly predict the number of spoken verbs in infants’ vocabulary at 24 months

Quantitative Explanation as a Tight Coupling of Data, Model, and Theory

What does it mean to explain data patterns? Cognitive psychologists and other scientists face this question when observable phenomena have to be explained in theoretical terms. Frequentist null-hypothesis testing – one prominent approach in psychology – controls error rates. Machine learning – an alternative prominent outside of, but not yet inside psychology – focuses on precise predictions. However, both alternatives often provide little insight into the data. We propose a combination of formal modeling and Bayesian statistical inference to ground explanations in data analysis. We support this approach by reference to philosophy of science and discussions of the current methods crisis in several empirical sciences and illustrate it with an example from visual attention research

How to learn the deictic shift through observation?

Krause F, Rohlfing K. How to learn the deictic shift through observation? Presented at the Second Workshop on Infant Language Development (WILD), Stockholm, Sweden

An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames

International audienceThe classic mapping metaphor posits that children learn a word by mapping it onto a concept of an object or event. However, we believe that a mapping metaphor cannot account for word learning, because even though children focus attention on objects, they do not necessarily remember the connection between the word and the referent unless it is framed pragmatically, that is, within a task. Our theoretical paper proposes an alternative mechanism for word learning. Our main premise is that word learning occurs as children accomplish a goal in cooperation with a partner. We follow Bruner's (1983) idea and further specify pragmatic frames as the learning units that drive language acquisition and cognitive development. These units consist of a sequence of actions and verbal behaviors that are co-constructed with a partner to achieve a joint goal. We elaborate on this alternative, offer some initial parametrizations of the concept, and embed it in current language learning approaches

Towards Tutoring an Interactive Robot

Wrede B, Rohlfing K, Spexard TP, Fritsch J. Towards tutoring an interactive robot. In: Hackel M, ed. Humanoid Robots, Human-like Machines. ARS; 2007: 601-612.Many classical approaches developed so far for learning in a human-robot interaction setting have focussed on rather low level motor learning by imitation. Some doubts, however, have been casted on whether with this approach higher level functioning will be achieved. Higher level processes include, for example, the cognitive capability to assign meaning to actions in order to learn from the tutor. Such capabilities involve that an agent not only needs to be able to mimic the motoric movement of the action performed by the tutor. Rather, it understands the constraints, the means and the goal(s) of an action in the course of its learning process. Further support for this hypothesis comes from parent-infant instructions where it has been observed that parents are very sensitive and adaptive tutors who modify their behavior to the cognitive needs of their infant. Based on these insights, we have started our research agenda on analyzing and modeling learning in a communicative situation by analyzing parent-infant instruction scenarios with automatic methods. Results confirm the well known observation that parents modify their behavior when interacting with their infant. We assume that these modifications do not only serve to keep the infant’s attention but do indeed help the infant to understand the actual goal of an action including relevant information such as constraints and means by enabling it to structure the action into smaller, meaningful chunks. We were able to determine first objective measurements from video as well as audio streams that can serve as cues for this information in order to facilitate learning of actions